The "useless" bayesian vision

The goal of maximum likelihood is to fit a distribution to some data.
Using the Bayes Theorem, we want to find the most likely value for the parametetrs of our model, given the data.

$$argmaxP(\theta |X)=argmax\frac{P(X|\theta )P(\theta )}{P(X)}$$Where:

• $P(\theta )$ is called the prior
• $P(X|\theta )$ is called the likelihood, which is not really a probability
• $P(\theta |X)$ is called the posterior

The likelyhood is equal to the probability density function of a gaussian, if we assume that the data was generated by a gaussian distribution.

Now the real stuff lol

We basically bruteforce fit a gaussian distribution on the data and then we get the one that maximizes the likelyhood function.

So we begin by fitting the gaussians, we start with $\mu =28$ and $\sigma =2$:

Which yields a likelyhood of:
$$L(\mathit{\mu },\mathbf{\Sigma };x)=\frac{1}{(2\pi {)}^{D/2}}{\frac{1}{|\mathbf{\Sigma }{|}^{1/2}}}^{-\frac{1}{2}(\mathbf{x}-\mathit{\mu }{)}^{\mathrm{T}}{\mathbf{\Sigma }}^{-1}(\mathbf{x}-\mathit{\mu })}=0.03$$

But we can do better right?
In fact, if we plug in $\mu =30$ and $\sigma =2$:

We get a likelyhood of 0.12, which is definitely better!

By the way, if we plot the likelyhood hovering all over the possible values of $\mu$, we can actually see that we can get the maximum likelyhood where the derivative of the whole thing is 0:

If we have multiple data points, the likelihood function will be the product of all the gaussians/individual likelyhood functions that are generated from the data points.

$$p(D;\mathit{\mu },\mathbf{\Sigma })=p(\{x_1,\dots ,x_N\};\mathit{\mu },\mathbf{\Sigma })=p(x_1;\mathit{\mu },\mathbf{\Sigma })\cdot p(x_2;\mathit{\mu },\mathbf{\Sigma })\dots \cdot p(x_N;\mathit{\mu },\mathbf{\Sigma })$$$$p(D;\mathit{\mu },\mathbf{\Sigma })=\prod _{i=1}^{N}p(x_i;\mathit{\mu },\mathbf{\Sigma })=\prod _{i=1}^N\mathcal{N}({\mathbf{x}}_i;\mathit{\mu },\mathbf{\Sigma })$$

So we take the derivative of this shit with respect to $\mu$ and we actually find the Maximum Likelihood parameters.

Obviously, you can do the same to find the standard deviations. You lock in the $\mu$ and you let the standard deviation change, then you stick with the value that gives the maximum likelyhood:

After endless mathematical proofs... the end

There is a whole ass proof to justify that the maximum likelihood estimate is equal to the mean of the measurements. The proof is covered in the link below:

Similarly, there is a proof that shows that the width of the distribution is equal to the standard deviation of the measurements:

These notions may be obvious, but now we have the math to back it up.